Work around changes to urllib.parse.urlsplit
#633
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for this contribution. I was wondering what's going on with those question marks... this PR makes it obvious where the problem was. However, I'm not sure if using the null character is the best way around it. It's risky, because the same thing that happened to the newline can happen to the null: the library implementation can change (I wonder why they did this, but it's out of our control) and then we'll be back to where we started. |
|
@judahrand I improved your PR slightly by replacing null with a configurable placeholder. @piskvorky Can you please have a look at this? Does this solution look sane to you? |
|
Merged! Thank you @judahrand for taking care of this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This attempts to fix the behaviour around question marks in URIs for S3 and GCS.
urllib.parse.urlsplitis used and a hack was put in place to replace?with\nto avoid issues withs params splitting. However, the behaviour ofurllib.parse.urlsplithas changed and now\n,\tand\rare all stripped from the URL before splitting.https://bugs.python.org/issue43882
My solution is to instead use the 'null character'. This could technically appear in a GCS Blob name... but should it? To me this feels fairrrrrllllly safe?
Checklist
Before you create the PR, please make sure you have:
Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.